Currently, as a 3D image viewing mode, generally, there is a mode (hereinafter, referred to as a two-viewpoint mode) in which two-viewpoint images alternately displayed are seen by wearing glasses of which a left-eye shutter is open at the time of displaying one image out of two-viewpoint images, and a right-eye shutter is open at the time of displaying the other image.
However, in such a two-viewpoint mode, a viewer needs to purchase glasses in addition to a 3D image display device, and accordingly, the viewer's willingness to buy reduces. In addition, since a viewer needs to wear glasses for viewing, it annoys the viewer. Accordingly, a demand for a viewing mode (hereinafter, referred to as a multi-viewpoint mode) increases in which a 3D image can be viewed without wearing glasses.
In the multi-viewpoint mode, multi-viewpoint images are displayed such that a viewable angle is different for each viewpoint, and, a 3D image can be seen by a viewer viewing images of arbitrary two viewpoints with left and right eyes without wearing glasses.
A display device that provides viewing of a multi-viewpoint mode, for example, generates multi-viewpoint images for a multi-viewpoint mode based on images of two viewpoints for a two-viewpoint mode and displays the generated multi-viewpoint images. More specifically, the display device acquires disparity (depth) of two-viewpoint images for a two-viewpoint mode using an image disparity estimating technology (Depth Estimation). Then, the display device generates a synthesized image of multi-viewpoint images adjacent to a viewpoint corresponding to the images of two viewpoints for a two-viewpoint mode using a multi-viewpoint image generating technology (View Generation) using the disparity between images of two viewpoints and a synthesis technology (View Synthesis) and displays the synthesized image.
Existing encoding modes include an advanced video coding (AVC) mode and a multi-view video coding (MVC) mode.
FIG. 1 is a diagram that illustrates an example of an image processing system that encodes a 2D image in accordance with the AVC mode and decodes the encoded image.
The image processing system 10 illustrated in FIG. 1 is configured by an imaging unit 11, an AVC encoder 12, and an AVC decoder 13.
The imaging unit 11 of the image processing system 10 captures an image A of a predetermined viewpoint and supplies the captured image to the AVC encoder 12. The AVC encoder 12 encodes the image A that is supplied from the imaging unit 11 in accordance with the AVC mode and supplies the encoded image to the AVC decoder 13. The AVC decoder 13 decodes the image A after encoding that is supplied from the AVC encoder 12 in accordance with a mode that corresponds to the AVC mode. The AVC decoder 13 outputs the image A that is acquired as a result of the decoding process, thereby displaying the image. In this way, the viewer can see a 2D image.
FIG. 2 is a diagram that illustrates an example of an image processing system that encodes an image of two viewpoints in accordance with the AVC mode and decodes the image.
The image processing system 20 illustrated in FIG. 2 is configured by an imaging unit 21A, an imaging unit 21B, an AVC encoder 22, and an AVC decoder 23.
The imaging unit 21A captures an image A of a predetermined viewpoint, halves the resolution of the image, and supplies a resultant image to the AVC encoder 22 as a left half image of the screen. In addition, the imaging unit 21B captures an image B of a viewpoint that is different from the viewpoint of the image A, halves the resolution of the image, and supplies a resultant image to the AVC encoder 22 as a right half image of the screen. The AVC encoder 22 encodes an image of which the left half is configured by the image A (hereinafter referred to as a half-resolution image A) of which the resolution has halved and of which the right half is configured by an image B (hereinafter, referred to as a half-resolution image B) of which the resolution has halved in accordance with the AVC mode and supplies the encoded image to the AVC decoder 23.
The AVC decoder 23 decodes the images after encoding that are supplied from the AVC encoder 22 in accordance with a mode that corresponds to the AVC mode. The AVC decoder 23 outputs images acquired as a result of the decoding process and alternately displays the half-resolution images A and B. At this time, a viewer, for example, wears glasses of which the left-eye shutter is open at the time of displaying the half-resolution image A and of which the right-eye shutter is open at the time of displaying the half-resolution image B, views the half-resolution image A only with the left eye, and views the half-resolution image B only with the right eye. In this way, the viewer can see a 3D image.
FIG. 3 is a diagram that illustrates an image processing system that encodes an image of two viewpoints in accordance with the MVC mode and decodes the encoded image.
The image processing system 30 illustrated in FIG. 3 is configured by an imaging unit 31A, an imaging unit 31B, an MVC encoder 32, and an MVC decoder 33.
The imaging unit 31A captures an image A of a predetermined viewpoint and supplies the captured image to the MVC encoder 32. In addition, the imaging unit 31B captures an image B of a viewpoint that is different from the viewpoint of the image A and supplies the captured image to the MVC encoder 32. The MVC encoder 32 encodes the image A that is supplied from the imaging unit 31A as a base image in accordance with the AVC mode and decodes the image B that is supplied from the imaging unit 31B as a dependent image in accordance with the MVC mode. The MVC encoder 32 supplies the images A and B after encoding to the MVC decoder 33.
The MVC decoder 33 decodes the images A and B after encoding that are supplied from the MVC encoder 32 in accordance with a mode that corresponds to the MVC mode. The MVC decoder 33 outputs the images A and B that are acquired as a result of the decoding process and alternately displays the images. At this time, a viewer, for example, wears glasses of which the left-eye shutter is open at the time of displaying the image A and of which the right-eye shutter is open at the time of displaying the image B, views the image A only with the left eye, and views the image B only with the right eye. In this way, the viewer can see a 3D image.
FIG. 4 is a diagram that illustrates an example of an image processing system that generates an image of multi-viewpoints by encoding an image of two viewpoints in accordance with the MVC mode and decoding the image.
The image processing system 40 illustrated in FIG. 4 is configured by imaging units 31A and 31B, an MVC encoder 32, an MVC decoder 33, a disparity detecting unit 41, and an image generating unit 42. In FIG. 4, the same reference numeral is assigned to a configuration that is the same as that illustrated in FIG. 3, and description thereof will not be appropriately presented.
Images A and B that are acquired as a result of decoding performed by the MVC decoder 33 are supplied to the disparity detecting unit 41 of the image processing system 40. The disparity detecting unit 41 detects the disparity of each pixel of the images A and B using the images A and B. The disparity detecting unit 41 generates a disparity image A′ that represents the disparity of each pixel of the image A and a disparity image B′ that represents the disparity of each pixel of the image B and supplies the generated disparity images to the image generating unit 42. The image generating unit 42 generates an image of multi-viewpoints of which the number corresponding to a display device not illustrated in the figure by using the images A and B acquired as a result of decoding performed by the MVC decoder 33 and the disparity images A′ and B′ supplied from the image generating unit 42. Then, the image generating unit 42 converts the resolution of the generated image of each viewpoint into “1/the number of viewpoints” of the resolution, synthesizes the images, and outputs the synthesized image so as to be displayed.
At this time, the multi-viewpoint images after the synthesis are displayed such that the viewing angles are different for each viewpoint, and a viewer can view a 3D image without wearing glasses by seeing images of arbitrary two viewpoints with his left and right eyes.
Meanwhile, a mode for encoding a multi-viewpoint image has also been devised (for example, see Patent Document 1).